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' Abstract 

A framework for linear-programming (LP) decoding of nonbinary linear codes over rings is de- 
veloped. This framework facilitates linear-programming based reception for coded modulation systems 
^ • which use direct modulation mapping of coded symbols. It is proved that the resulting LP decoder has 

^ ' the 'maximum-likelihood certificate' property. It is also shown that the decoder output is the lowest cost 

I pseudocodeword. Equivalence between pseudocodewords of the linear program and pseudocodewords of 



> 



o 



X 



graph covers is proved. It is also proved that if the modulator-channel combination satisfies a particular 



■ symmetry condition, the codeword error rate performance is independent of the transmitted codeword, 

■"sj" ' Two alternative polytopes for use with linear-programming decoding are studied, and it is shown that 

o . 

00 , representations lead to polynomial-time decoders for a wide variety of classical nonbinary linear codes. 



for many classes of codes these polytopes yield a complexity advantage for decoding. These polytope 



LP decoding performance is illustrated for the [11,6] ternary Golay code with ternary PSK modulation 
over AWGN, and in this case it is shown that the performance of the LP decoder is comparable to 



^ I codeword-error-rate-optimum hard-decision based decoding. LP decoding is also simulated for medium- 

length ternary and quaternary LDPC codes with corresponding PSK modulations over AWGN. 
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I. Introduction 



Low-density parity-check (LDPC) codes [[11 have become very popular in recent years due to 
their excellent performance under sum-product (SP) decoding (or message-passing decoding). 
The primary research focus in this area to date has been on binary LDPC codes. Finite-length 
analysis of such LDPC codes under SP decoding is a difficult task. An approach to such an 
analysis was proposed in [[21 based on the consideration of so-called pseudocodewords and their 
pseudoweights, defined with respect to a structure called the computation tree. By replacing this 
set of pseudocodewords with another set defined with respect to cover graphs of the Tanner 
graph (here called graph-cover pseudocodewords), the analysis was found to be significantly 
more tractable while still yielding accurate experimental results [[3l, [ffl, [[51. 

In [[6l and [[3, the decoding of binary LDPC codes using linear-programming (LP) decoding 
was proposed, and many important connections between linear-programming decoding and clas- 
sical message-passing decoding were established. In particular, it was shown that the LP decoder 
is inhibited by a set of pseudocodewords corresponding to points in the LP relaxation polytope 
with rational coordinates (here called linear-programming pseudocodewords), and that the set of 
these pseudocodewords is equivalent to the set of graph-cover pseudocodewords. This represents 
a major result as it indicates that essentially the same phenomenon determines performance of 
LDPC codes under both LP and SP decoding. 

For high-data-rate communication systems, bandwidth-efficient signalling schemes are re- 
quired which necessitate the use of higher-order (nonbinary) modulation. Of course, within such a 
framework it is desirable to use state-of-the-art error-correcting codes. Regarding the combination 
of LDPC coding and higher-order modulation, bit-interleaved coded modulation (BICM) [[H is 
a high-performance method which cascades the operations of binary coding, interleaving and 
higher-order constellation mapping. Here however the problem of system analysis is exacerbated 
by the complication of joint design of binary code, interleaver and constellation mapping; this 
becomes even more difficult when feedback is included from the decoder to the demodulator [9[. 

Alternatively, higher-order modulation may be achieved in conjunction with coding by the 
use of nonbinary codes whose symbols map directly to modulation signals. A study of such 
codes over rings, for use with PSK modulation, was performed in [[TOl . with particular focus on 
the ring of integers modulo 8. Nonbinary LDPC codes over fields have been investigated with 
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direct mapping to binary [fTH and nonbinary [[T2ll . |fT3l . lfT4]| . IfTSl modulation signals; in all of 
this work, SP decoding (with respect to the nonbinary alphabet) was assumed. Recently, some 
progress has been made on the topic of analysis of such codes; in particular, pseudocodewords 
of nonbinary codes were defined and some bounds on the pseudoweights were derived lfT6l . 

In this work, we extend the approach in I7J towards coded modulation, in particular to codes 
over rings mapped to nonbinary modulation signals. As was done in [7J, we show that the 
problem of decoding may be formulated as an LP problem for the nonbinary case. We also 
show that an appropriate relaxation of the LP leads to a solution which has the 'maximum- 
likelihood (ML) certificate' property, i.e. if the LP outputs a codeword, then it must be the ML 
codeword. Moreover, we show that if the LP output is integral, then it must correspond to the ML 
codeword. We define the graph-cover pseudocodewords of the code, and the linear-programming 
pseudocodewords of the code, and prove the equivalence of these two concepts. This shows that 
the links between LP decoding on the relaxation polytope and message-passing decoding on the 
Tanner graph generalize to the nonbinary case. Of course, while we use the term 'nonbinary' 
throughout this paper, our framework includes the binary framework as a special case. 

For coded modulation systems using maximum-likelihood (ML) decoding, the concept of 
geometric uniformity ifTTl was introduced as a condition which, if satisfied, guarantees codeword 
error rate (WER) performance independent of the transmitted codeword (this condition was used 
for design of the coded modulation systems in ifTOl ). An analogous symmetry condition was 
defined in [fTSll for binary codes over GF{2) with SP decoding; this was later extended to 
nonbinary codes over GF{q) by invoking the concept of coset LDPC codes [[T3l . [fT4l . We show 
that for the present framework, there exists a symmetry condition under which the codeword error 
rate performance is independent of the transmitted codeword. This provides a condition somewhat 
akin to geometric uniformity for the present framework. It is noteworthy that the same symmetry 
condition has recently been shown to yield codeword-independent decoder performance in the 
context of SP decoding [|T9l and also in the context of ML decoding [|20l . In particular, this 
identifies a 'natural' mapping for nonbinary codes mapped to PSK modulation, where LP, SP 
or ML decoding is used with direct modulation mapping of coded symbols. 

For the binary framework, alternative polytope representations were studied which gave a 
complexity advantage in certain scenarios jH, [|71, [|2TI . [|22l . [|23l . Analogous to these works, 
we define two alternative polytope representations, which offer a smaller number of variables 
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and constraints for many classes of nonbinary codes. We compare these representations with 
the original polytope, and show that both of them have equal error-correcting performance to 
the original LP relaxation. Both of these representations lead to polynomial-time decoders for a 
wide variety of classical nonbinary linear codes. 

To demonstrate performance, LP decoding is simulated for the ternary Golay code mapped 
to ternary PSK over AWGN, and the LP decoder is seen to perform approximately as well as 
codeword-error-rate optimum hard-decision decoding, and approximately L5 dB from the union 
bound for codeword-error-rate optimum soft-decision decoding. 

The paper is organized as follows. Section |II] introduces general settings and notation. The 
nonbinary decoding problem is formulated as a linear-programming problem in Section |llll and 
basic properties of the decoding polytope are studied in Section |IVl A sufficient condition 
for codeword-independence performance of the decoder is presented in Section |Vl Linear- 
programming pseudocodewords are defined in Section |Vll and their properties are discussed. 
Their equivalence to the graph-cover pseudocodewords is shown in Section IVIII Two alternative 
polytope representations are presented in Sections IVIIII and |Kl both of which have equivalent 
performance to the original but may provide lower-complexity decoding. Simulation results are 
presented in Section |X] for some example coded modulation systems. Finally, some directions 
for future research are proposed in Section |Xll 

IL General Settings 

We consider codes over finite rings (this includes codes over finite fields, but may be more 
general). Denote by 9^ a ring with q elements, by its additive identity, and let = 9^\{0}. 
Let C be a code of length n over the ring defined by 

C = {ceD^" : cn^ = 0} (1) 

where 7i is an m x n matrix (with entries from IK) called the parity-check matrix of the code C. 
Obviously, the code C may admit more than one parity-check matrix; however we will consider 
that the parity-check matrix Ti is fixed in this paper. 

Linearity of the code C follows directly from ([U). Also, the rate of the code C is defined as 
R{C) = log^(|C|)/?2 and is equal to the number of information symbols per coded symbol. The 
code C may then be referred to as an [n, logg(|C|)] linear code over 

4 



Denote the set of column indices and the set of row indices of7-^byX = {1,2, - ■ ■ ,n} and 
J' = {1,2, ■■ ■ ,m}, respectively. We use notation Hj for the j-th row of H, where j E J . 
Denote by supp(c) the support of a vector c. For each j G J , let Xj = supp(7ij) and dj = 
and let d = maxj^ j{dj}. 

Given any c G 9^", we say that parity-check j G ^7 is satisfied by c if and only if 

cnJ = Y,Ci■^J,^ = o■ (2) 

For j G define the single parity-check code Cj over 9^ by 

Note that while the symbols of the codewords in C are indexed by X, the symbols of the 
codewords in Cj are indexed by Ij. We define the projection mapping for parity-check j E J 
by 

Xj{c) = (Ci),ej. 

Then, given any c G IK", we may say that parity-check j E J' is satisfied by c if and only if 

x,{c) E C, , (3) 

since © and ([3]) are equivalent. Also, it is easily seen that c G C if and only if all parity-checks 
J G J7 are satisfied by c. In this case we say that c is a codeword of C. 

We shall take an example which shall be used to illustrate concepts throughout this paper. 
Consider the [4, 2] linear code over 9^ = Z3 with parity-check matrix 

/ 1 2 2 1 \ 

n=\ (4) 

12 12/ 

Here Ji = {1, 2, 3, 4}, X2 = {1, 3, 4}, and the two single parity-check codes Ci and C2, of length 
di = A and (i2 = 3 respectively, are given by 

Ci = {(61 62 h hi) : 61 + 262 + 263 + &4 = 0} 

and 

C2 = {ihhb^) : 261 + 63 + 264 = 0} . 



III. Decoding as a Linear-Programming Problem 

Assume that the codeword c = (ci, C2, ■ ■ ■ , c„) EC has been transmitted over a g-ary input 
memoryless channel, and a corrupted word y = (yi,y2, ■ ■ ■ ,?/„) G S" has been received. Here 
E denotes the set of channel output symbols; we assume that this set either has finite cardinality, 
or is equal to or for some integer / > 1. In practice, this channel may represent the 
combination of modulator and physical channel. We assume hereafter that all information words 
are equally probable, and so all codewords are transmitted with equal probability. 

It was suggested in f6\ to represent each symbol as a binary vector of length where the 

entries in the vector are indicators of a symbol taking on a particular value. Below, we show 
how this representation may lead to a generalization of the framework of [17| to the case of 
nonbinary coding. This generalization is nontrivial since, while such a representation converts 
the nonbinary code into a binary code, this binary code is not linear and therefore the analysis 
in O, (3 is not directly applicable. 

For use in the following derivation, we shall define the mapping 

^ : m ^ {0,1}^"^ C R'-^ , 

by 

= x = (x(^))^e5H- , 

such that, for each 7 G 

1 if 7 = a 
otherwise. 

We note that the mapping ^ is one-to-one, and its image is the set of binary vectors of length 
q — I with Hamming weight or 1 . Building on this, we also define 

H : 9^" — > {0, c , 

according to 

S(c) = (^(ci) I ^(C2) I ■■■ |^(c„)). 

We note that S is also one-to-one. 

Now, for vectors / G M*^"^"^^", we adopt the notation 

/ = (/l I /2 I ■ ■ ■ I fn) 5 
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where 

Vi e X, /, = (/f ^)«eiH- ■ 
Also, we may use this notation to write the inverse of H as 

We also define a function A : S — ^ (M U {±00})''"^ by 

where, for each 1/ e E, a e 

and p{y\c) denotes the channel output probability (density) conditioned on the channel input. 
We extend this to a map on E" by defining A{y) = {X{yi) \ \{y2) I • • • I 

The codeword-error-rate-optimum receiver operates according to the maximum a posteriori 
(MAP) decision rule: 

c = argmaxp( c\y) 

p{y\c)p{c) 

— arg max 



cGC p{ y ) 

Here p{-) denotes probability if S has finite cardinality, and probability density if S has infinite 
cardinality. 

By assumption, the a priori probability p(c) is uniform over codewords, and p{y) is indepen- 
dent of c. Therefore, the decision rule reduces to maximum-likelihood (ML) decoding: 

c = argmaxp( y \ c) 



= argmaxTTp(yi|ci) 

i=l 

n 

= argm^^log(p(|/i|Q)) 

i=l 

— argmm > log —. — 1 — r 
-ec^ ^\p{yi\ci) 

n 

= argmin^A(yi)£(ci)^ 



cGC 

i=l 



argmin A(t/)H(c)'^ , 



where we have made use of the memoryless property of the channel, and of the fact that if 
Q = a G IH", then A(?/j)^(cj)^ = X'^^^yi). This is then equivalent to 

where 

/ = arg min A(t/)/^ , (5) 

/e/c(C) 

and /C(C) represents the convex hull of all points / G M^"?^^)" which correspond to codewords, 
i.e. 

/C(C) = /7conv{3(c) : ceC] . 

Therefore it is seen that the ML decoding problem reduces to the minimization of a linear 
objective function (or cost function) over a polytope in M*^"^^^)". The number of variables and 
constraints for this linear program is exponential in n, and it is therefore too complex for practical 
implementation. To circumvent this problem, we formulate a relaxed LP problem, as shown next. 
The solution we seek for / (i.e. the desired LP output) is 

/ = H(c) = (^(ci) I ^(C2) I . . . I ^(c„)) . (6) 

Note that ^ implies that the solution we seek for each f^{iET) is an indicator function which 
"points" to the i-th transmitted symbol q, i.e. 

1 if a = Cj 

otherwise. 

We now introduce auxiliary variables whose constraints, along with those of the elements of /, 
will form the relaxed LP problem. We denote these auxiliary variables by 

Wj^b for j e J,be Cj , 
and we denote the vector containing these variables as 

= (^j)jej ^^^^^ = i^i'bj^^c, ^^^^ ■ 
The solution we seek for these variables is 

1 if 6 = xAc] 

VJGJT : w,^b={ ' (7) 

otherwise. 



VzeX : 



Note that the solution we seek for each Wj (j E J) is an indicator function which "points" to 
the j-th transmitted local codeword x^{c). Based on ([7]), we impose the constraints 

Vj G J, Vb G Cj, Wj^b > , (8) 

and 

Vjej, 5^w7,-5 = l. (9) 

Finally, we note that the solution we seek (given by the combination of ([6]) and dV])) satisfies 
the further constraints 

/f^ = E5ec,,..=a^.>- (10) 

It is interesting to note that from ([8]) and dH), each vector Wj (for j E J') may be interpreted as 
a probability distribution for the local codeword b E Cj, in which case each (for i E I) has a 
natural interpretation (via (fTOl) ) as the corresponding probability distribution for the i-th coded 
symbol Cj E 91. The following example illustrates the connection (flOl) between / and w. 

Example 3.1: Consider the example [4,2] code over Z3 defined by the parity-check ma- 
trix dH). The second row 7-^2 of the parity-check matrix corresponds to the parity-check equation 

2bi + 63 + 264 = 

over Z3. Here b = (61 63 64) E C2. Assume that the values of W2,b for b E C2 are as given in 
the following table. 



616364 




616364 


W2,b 


616364 


W2,b 


000 


0.01 


102 


0.05 


201 


0.15 


oil 


0.04 


110 


0.07 


212 


0.32 


022 


0.05 


121 


0.08 


220 


0.23 



Then, some of the values of f-"' are as follows: 

/f) = 0.15 + 0.32 + 0.23 = 0.7; 

= 0.04 + 0.07 + 0.32 = 0.43; 
/f ) = 0.05 + 0.05 + 0.32 = 0.42 . 
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Constraints (l8])-(fT0l) may be interpreted as the statement that for all j E J', the vector fj = 
{fi)ieJj li^s in the convex hull K,{Cj). Constraints (|8l)-(fT0l) form a polytope which we denote 
by Q. The minimization of the objective function ([5]) over Q forms the relaxed LP decoding 
problem. This LP is defined by 0{qn+q'^m) variables and 0{qn+q'^m) constraints, and therefore, 
the number of variables and of constraints scales as approximately q^. 

We note that the further constraints 

\fj eJ,ybeCj, wj-6<l, (11) 

Vi G J, Va G 0< fj""^ <1. (12) 

and 

Vz G J, Yl f^'^^ ^ 1 ■ (13) 

foUow from the constraints (l8l)-(fT0l). for any (/, w) G Q. 

Now we may define the decoding algorithm, which works as follows. The decoder solves 
the LP problem of minimizing the objective function ([5]) subject to the constraints (l8l)-(fT0l). If 
/ G {0, the output is the codeword 3^^(/) (we shall prove in the next section that 

this output is indeed a codeword). This codeword may then be the correct one (we call this 
'correct decoding') or an incorrect one (we call this 'incorrect decoding'). If / ^ {0, 
the decoder reports a 'decoding failure'. Note that in this paper, we say that the decoder makes 
a codeword error when the decoder output is not equal to the transmitted codeword (this could 
correspond to a 'decoding failure', or to an 'incorrect decoding'). 

The time complexity of an LP solver depends on the number of variables and constraints in 
the LP problem. The simplex method is a popular and practically efficient algorithm for solving 
LP problems. However, its worst-case time complexity has been shown to be exponential in 
the number of variables. There are other known LP solvers, such as solvers that are based on 
interior-point methods [|24l Chapter 11], which have time complexity polynomial in the number 
of variables and constraints. For more detail the reader may also refer to [l25l. We note, however, 
that the standard iterative decoding algorithms (such as the min-sum or sum-product algorithms) 
have time complexity which is linear in the block length of the code, and therefore significantly 
outperform the LP decoder in terms of efficiency. 
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IV. PoLYTOPE Properties 

The analysis in this section is a direct generalization of the results in [7J. 

Definition 4.1: An integral point in a polytope is a point with all integer coordinates. 
Proposition 4.1: 

1) Let (/, w) e Q, and /^"^ G {0, 1} for every i G T, « G ^K". Then H-^(/) G C. 

2) Conversely, for every codeword c G C, there exists w such that (/, w) is an integral point 
in Q with / = 3(c). 

Proof: 

1) Suppose (/, w) G Q, and //"^ G {0, 1} for every z G J, a G 9^". 

Let c = S^^(/); by (fT3l) . this is well defined. Now, fix some j E J' and define t = Xj{c). 
Note that from these definitions it follows that for any i el, a e //"^ = 1 if and 
only if U = a. Now let r G Cj, r ^ t. Since r and t are distinct, there must exist a G 
and / G Tj such that either ri = a and 7^ a, or = a and r/ 7^ a. We examine these 
two cases separately. 
• If = a and t/ 7^ a, then by (flOl) 



Therefore Wj^b = for all b G with 6^ = a, and in particular Wj^r = 0. 
If = a and ri ^ a, then by © and (flOl) 



= 1-/, 



beCj beCj,b,=a 

Yl ^^'^ ■ 

beCj, bi^a 

Therefore Wj^b = for all b G Cj with bi 7^ a, and in particular Wj^r = 0. 
It follows that Wj^r = for all r e Cj, r ^ t. But by ® this implies that t G (and that 
Wj^t = !)• Applying this argument for every j E J implies c G C. 
2) For c G C, we let / = S(c). For each parity-check j G J", we let t = Xj[c) G Cj and 
then set 

1 \fb = t 

Vj G : Wj- 6 



otherwise. 
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It is easily checked that the resulting point (/, w) is integral and satisfies constraints dH)- 

m. 

u 

The following proposition assures the so-called ML certificate property. 

Proposition 4.2: Suppose that the decoder outputs a codeword c G C. Then, c is the 
maximum-likelihood codeword. 

The proof of this proposition is straightforward. The reader can refer to a similar proof for 
the binary case in [|71. 

V. Codeword-Independent Decoder Performance 

In this section, we state and prove a theorem on decoder performance, namely, that under a 
certain symmetry condition, the probability of codeword error is independent of the transmitted 
codeword. The proof generalizes the corresponding proof for the binary case which may be 
found in 0. 

Symmetry Condition. 

For each /3 G IK, there exists a bijection 

Tp : S — ^ S , 

such that the channel output probability (density) conditioned on the channel input satisfies 

p{y\a) = p{Tp{y)\a - (3) , (14) 

Vy G S, a G 9^. When S is equal to or C' for / > 1, the mapping is assumed to be 
isometric with respect to Euclidean distance in S, for every /3 G 9^. 

Note that the symmetry condition above is very similar to that introduced in [|20ll which 
guarantees codeword-independent performance under ML decoding. 

Theorem 5.1: Under the stated symmetry condition, the probability of codeword error is 
independent of the transmitted codeword. 

Proof: We shall prove the theorem for the case where E has infinite cardinality; the case of 
discrete S may be handled similarly. Fix some codeword c G C, c 7^ 0. We wish to prove that 

Pr(Err | c) = Pr(Err | 0) , 
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where Pr(Err | c) denotes the probability of codeword error given that the codeword c was 
transmitted. 
Now 

Pr(Err | c) = Pr(y E B{c) \ c) , 

where 

B(c) = {yGS" : 3{f,w)eQJ^S{c) 

with A(y)/^ < Aiy)S{cf} . 
Here B(c) is the set of all received words which may cause codeword error, given that c was 
transmitted. Recall that the elements of A{y) are given by 

A<'.)(,.) = log ( . (15) 



p{yi\a) 

for i e I, a e Also 

Pr(Err | 0) = Pr{y e B{0) \ 0) 

where 

i?(0) = {yGS" : 3Cf,w)eQ,f^EiO) 

with Aiy)f < A{y)S{Of} . 

So we write 



Pr(Err | c) = / p{ y \ c) dy (16) 

JyeB{c) 

and 

Pr(Err | 0) = / p{y\0)dy. (17) 

JyeB{0) 

Now, setting a = P in the symmetry condition ([14)) yields 

Piy\P)=PiMym (18) 

for any ?/ G S, /3 G 91. 

We now define G : S" — > S" and y as follows. 

1/ = G{y) s.t. Miel : jji = Tp{yi) where /5 = q . 

We note that G is a bijection from the set S" to itself, and that if y, z G and (3 = Ci then 
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and so 



\G{y)-G{z) 



\y - 2| 



i.e. G is isometric with respect to Euclidean distance in S"^. 

We prove that the integral (fT6l) may be transformed to (fTTI) via the substitution y 
First, we have 



G{y). 



p{y\c) = Ylp{yi\ci) 

= n n p(y^\f^^ 

l3einieJ,Ci=i3 

= n n pi^y^M 

l3einieJ,Ci=i3 

= n n 

= Up^m 

= P{lf\0). 

Since G is isometric with respect to Euclidean distance in S'^, it follows that the Jacobian 
determinant of the transformation is equal to unity. Therefore, to complete the proof, we need 
only show that 

y G B{c) if and only if y e B{0) . 

We begin by relating the elements of A{y) to the elements of A(^). Let i E X, a E 
Suppose Ci = (3 eD\. We then have 

= log 
= log 





1 




)|- 


piMvi) 


\a - 


PiVil - 




p{yi\a - 


-P) 



This yields 



A(")(y,) if/5 = 

-A(-")(y,) if a = [3 

A^""^) [Vi] - A(-^) {vi) otherwise. 
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Next, for any point (/, it>) G Q we define a new point (/, w) as follows. For (3 = Ci and all 
i el, a e 9^", 

ji^) ^ 1 1 - E,.^- fV' if « = -/? ^^^^ 

1 fi^^^^ otherwise. 

For all j E J, r E Cj we define 
where 

6 = r + Xj{c) . 

Next we prove that for every (/, w) E Q, the new point (/, w) lies in Q and thus is a 
feasible solution for the LP. Constraints ([8]) and ^ obviously hold from the definition of ib. To 
verify (flOl) . we let j E J, i E Ij and a E . We also let /3 = Ci. We now check two cases: 

. If a = —f3, 

= Yl ^i'*' 

beCj 7eiH- beCj, 6^=7 



E 

E 

r&Cj, ri=a 



If a ^ 



E 



Therefore (/, w) E Q, i.e. (/, w) is a feasible solution for the LP. We write (/, w) = L{f, w). 
We also note that the mapping L is a bijection from Q to itself; this is easily shown by verifying 
the inverse 

^ I 1 - f''' if " = (20) 

fj;"'~^^ otherwise 
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for aWi E I, a e ^ , and 



where 



for all j E J, b E Cj. 

We now prove that for every (/, w) E Q, (/, w) = L{f, w) satisfies 

Aiy)f - A(y)H(c)^ = Aiy)f - A(y)H(0)^ . 



(21) 



We achieve this by proving 



(22) 



for every i EX. We may then obtain (|2T]) by summing (|22l) over i EX. Let (3 = Ci. We consider 
two cases: 

. If ^ = 0, dill) becomes 



which holds since A^"^(yj) = X^°'\yi) and //"^ = f^"^ for all a E in this case. 
. If/5^0, 

A(2/,)/r-A(2/.)^(Q)^ 



where we have made use of the substitution a = j — P in the third line. Therefore (|22l) holds, 
proving (|2T]) . 



77^/3 
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Finally, we note that it is easy to show, using (fT9l) and (|20l ). that / = H(c) if and only if 
/ = S(0). Putting together these results, we may make the following statement. Suppose we 
are given y,y E S" with y = G{y). Then the point {f,w) E Q satisfies / ^ S(c) and 
A{y)f^ < A{y)'E{c)^ if and only if the point {f,w) = L{f,w) E Q satisfies / ^ S(0) and 
^{y)f < -^(i/)S(0)'^. This statement, along with the fact that both G and L are bijective, 
proves that 



We next provide, with details, some examples of modulator-channel combinations for which 
the symmetry conditions hold. 

Example 5.1: Discrete memoryless q-ary symmetric channel. Here we denote the ring el- 
ements by = {ao, ai,-- - ,aq_i}. Also S = {so,si,--- where the channel output 
probability conditioned on the channel input satisfies, for each t, A; G {0, 1, ■ ■ ■ , g — 1}, 



where p represents the probability of transmission error. Here we may define the mapping 
for each (3 Ed\ according to 



for all t G {0, 1, ■ ■ ■ g — 1}. It is easy to check that these mappings are bijective and satisfy the 
symmetry condition. 

Example 5.2: Orthogonal modulation over AWGN. Here S = W, and denoting the ring 
elements by 9^ = {ao,ai, ■ ■ ■ ,ag_i}, the modulation mapping may be written without loss of 
generality as 



y E B{c) if and only if y E B{0) . 




if t = k 
p/{q — 1) otherwise 



T/3(si) = Si where = - /3 



M 



such that, for each A; = 0, 1, ■ ■ ■ , g — 1, 



M{ak) 



= X = 



where 
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Here we may define the mapping rg for each (3 e according to (where y = {y'^'^\y^^\ ■ ■ ■ ,y^'^ G 

My) = ^ 

such that for each / G {0, 1, ■ ■ ■ , g — 1}, 

/i) ^ y{k) ^hej-e = ai + (3. 

It is easily checked that these mappings are bijective and isometric, and satisfy the symmetry 
condition. 

Example 5.3: q-ary PSK modulation over AWGN. 
Here S = C, and again denoting the ring elements by 9^ = {ao, cti, ■ ' ' ; the modulation 

mapping may be written without loss of generality as 

M ■ 9^^C 

such that 

M(a,) = exp { ^) (23) 



for = 0, 1, ■ ■ ■ , g — 1 (here i = v^—T). Here (fTSi) . together with the rotational symmetry of the 
g-ary PSK constellation, motivates us to define, for every P = ak E 9^, 

(—t27[k\ 
— - — J ■ X Vx G C (24) 

Next, we also impose the condition that 9^ under addition is a cyclic group. To see why we 
impose this condition, let a = G 9^ and /3 = a; G 9^. By the symmetry condition we must 
have 

p{yi\a + (3) = p{Ta+i3{yi)\0) 

and also 

p{yi\a + (3) = p{T,3{yi)\a) 

= PiraMy.m. 

In order to equate these two expressions, we impose the condition Ta+f3{x) = rQ,(r^(x)) for all 
X G C, a, /5 G 9^. Letting a + /3 = G 9^, and using (l24l) yields 

f-l2^k\ f-t2TTl\ f-t27Tp\ 

exp ■ exp = exp 

\ Q J \ Q J \ Q J 
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and thus p={k + l) mod q. 
Therefore, we must have 

mod q 

(25) 

for all Ofc, ai G This implies that 9^, under addition, is a cyclic group. 

It is easy to check that the condition that £H under addition is cyclic, encapsulated by (|25l) . 
along with the modulation mapping (|23l) . satisfies the symmetry condition, where the appropriate 
mappings rg are given by (|24)) . This means that codeword- independent performance is guaranteed 
for such systems using nonbinary codes with PSK modulation. This applies to AWGN, flat 
fading wireless channels, and OFDM systems transmitting over frequency selective channels 
with sufficiently long cyclic prefix. 

VI. Linear Programming Pseudocodewords 

Definition 6.1: A linear-programming pseudocodeword (LP pseudocodeword) of the code 
C, with parity-check matrix H, is a pair [h, z) where h G M^'^^^^" and 



( Zj.b ) 



where Zj^t, is a nonnegative integer for all j E J^, b E Cj, such that the following constraints are 
satisfied: 

Vj Ej,yiE Ij, V« E 

and 

bee, 

where M is a nonnegative integer independent of j. 

It follows from (|26|) that hf^^ is a nonnegative integer for dX\ i E X, a E . We note that the 
further constraints 

Vj Ej.^hE Cj, Zj,b < M , (28) 

Vz e T, Va e m-, < /ij"^ < M , (29) 
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and 

foUow from the constraints (|26l ) and (|27] ). 
For each i el, we also define 



,(0) _ A# _ ^ ;,(«) 

,(0) 



hf^ = M- J2 ■ (31) 



By (|30l) . /i- is a nonnegative integer for all i El. Now, for any j E J, i E Ij we have 

= 5Z 5Z ^J''' 

where we have used ( |26l ) and (|27] ). 

Corresponding to the LP pseudocodeword (h, z) defined above, we define the normalized LP 
pseudocodeword as the vector obtained by scaling of {h,z) by a factor 1/M. We also define 
the n X g LP pseudocodeword matrix 

H= fe") 



The normalized LP pseudocodeword matrix is defined as (1/M) ■ H. 

Note that if we interpret {zj^b/M} (for each j G JT) as a probability distribution for the local 
codeword b E Cj, then the i-th row of the normalized LP pseudocodeword matrix (for i E T) can 
be interpreted as the corresponding probability distribution for the z-th coded symbol Cj E 9^. 
This idea of interpretating pseudocodewords as probability distributions was used in |l3l for the 
binary case. 

Example 6.1: As an illustration, we provide an LP pseudocodeword for the example [4, 2] 
code over Z3 defined by the parity-check matrix (51). The reader may check that 

{h^\h^\hf,h^l^) = {2 2 2 2) (32) 

and 

(/iS'^/if ,/if) = (2 2 0) (33) 
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together with 



Zl,b 



2 if 6 = (2 1 1 0) 
2 if 6 = (1 2 1) 
otherwise. 



(34) 



and 



2 if 6 = (2 1) 
Z2,b = 2 if 6 = (1 1 0) 
otherwise, 

satisfy and dSV]), where M = 4 in We also obtain from (EB 

{h?\hf\hf,h^f^) = {0022). 

Therefore (I32l)-(l35l) define an LP pseudocodeword, with pseudocodeword matrix 

/ 2 2 \ 
2 2 
2 2 
\ 2 2 / 

The corresponding normalized LP pseudocodeword matrix is then given by 



(35) 



H 



(36) 



(o 


1 

2 







1 


1 


2 


2 


1 


1 





2 


2 


u 


1 
2 


) 



(37) 



Here the probabilistic interpretation of this normalized LP pseudocodeword matrix corresponds 
to an equiprobable distribution of symbols from {1,2} for the first two symbols in the codeword, 
and an equiprobable distribution of symbols from {0, 1} for the last two symbols in the codeword. 
Theorem 6.1: Assume that the all-zero codeword was transmitted. 

1) If the LP decoder makes a codeword error, then there exists some LP pseudocodeword 
{h, z), h^O, such that A{y)h^ < 0. 

2) If there exists some LP pseudocodeword {h, z), h ^ 0, such that A(y)h^ < 0, then the 
LP decoder makes a codeword error. 

Proof: The proof follows the lines of its counterpart in 
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1) Let {f,w) be the point in Q which minimizes A{y)f'^. Suppose there is a codeword 
error; then / 7^ 0, and we must have A{y)f^ < 0. 

Next, we construct the LP pseudocodeword (h, z) as follows. Since the LP has rational 
coefficients, all elements of the vectors / and w must be rational. Let M denote their 
lowest common denominator; since / 7^ we may have M > 0. Now set /ij-"-* = M ■ 
for all i e2, a E and set zj^b = M ■ wj^b for all j E J and b E Cj. 
By ©-Go]), {h, z) is an LP pseudocodeword and h since / 7^ 0. Also A{y)f'^ < 
implies A{y)hF < 0. 

2) Now, suppose that an LP pseudocodeword {h, z) with 7^ satisfies A{y)hF < 0. Since 
h, 7^ we have M > in Now, set /^°^ = hf^ /M for alH G J, a G 9^", and set 
Wj^b = Zj^b/^I for all j E J and h E Cj. It is straightforward to check that (/, w) satisfies 
all the constraints of the polytope Q. Also, h 7^ implies / 7^ 0. Finally, A{y)h^ < 
implies A{y)f'^ < 0. Therefore, the LP decoder will make a codeword error. 

■ 

VIL Equivalence Betvv^een Pseudocodeword Concepts 
A. Tanner Graphs and Graph-Cover Pseudocodewords 

The Tanner graph of a linear code C over 9^ is an equivalent characterization of the code's 
parity-check matrix H. The Tanner graph Q = {V,£) has vertex set V = {mi,M2, ■ ■ ■ ,m„} U 
{fi, f2, ■ ■ ■ ; ^m}, and there is an edge between Ui and Vj if and only if Hj^i 7^ 0. This edge is 
labelled with the value We denote by ^{v) the set of neighbors of a vertex v eV. 

For any word c = (ci,C2,--- , c„) E 91", the Tanner graph allows an equivalent graphical 
statement of the condition c E Cj for each j E J, as follows. The variable vertex ui is labelled 
with the value q for each i E I. Equation ^ (or ([3])) is then equivalent to the condition 
that for vertex Vj, the sum, over all vertices in J\f{vj), of the vertex labels multiplied by the 
corresponding edge labels is zero. This graphical means of checking whether a parity-check is 
satisfied by c G 9^" will be useful when defining graph-cover pseudocodewords later in this 
section. 

To illustrate this concept. Figure [T] shows the Tanner graph for the codeword c = (1 2 1) of 
the example [4, 2] code over Z3 defined by the parity-check matrix dH). In Figure [B edge labels 
are shown in square brackets, and vertex labels in round brackets. The reader may check that for 
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Vi V2 




Ui{l) ^2(0) ^3(2) ^4(1) 

Fig. 1. Tanner graph for the example [4, 2] code over Z3. Edge labels are shown in square brackets, and vertex labels in round 
brackets. For each parity-check j, the sum, over all vertices in Af{vj), of the vertex labels multiplied by the corresponding edge 
labels is zero; therefore all parity-checks are satisfied. 

each parity-check j = 1,2, the sum, over all vertices in J\f{vj), of the vertex labels multiplied 
by the corresponding edge labels is zero. 

We next define what is meant by a finite cover of a Tanner graph. 

Definition 7.1: ([4J) A graph Q = (V, £) is a finite cover of the Tanner graph Q = (V, £) if 
there exists a mapping 11 : V — > V which is a graph homomorphism (11 takes adjacent vertices 
of Q to adjacent vertices of Q), such that for every vertex v E Q and every v E Il^^{v), the 
neighborhood J\f{v) of v (including edge labels) is mapped bijectively to J\f{v). 

Definition 7.2: ([4J) A cover of the graph Q is said to have degree M, where M is a positive 
integer, if |n"^(t>)| = M for every vertex v eV. We refer to such a cover graph as an M -cover 

of g. 

Fix some positive integer M. Let Q = (V, £) be an M-cover of the Tanner graph Q = (V, £) 
representing the code C with parity-check matrix H. The vertices in the set 11"^ (wj) are called 
copies of Ui and are denoted {ui,i,Uj,25 ■ ■ ■ ^Wj^Af}, where i E I. Similarly, the vertices in the 
set Il^^{vj) are called copies of Vj and are denoted f j,2, ■ ■ ■ , ^j.a/}. where j E J . 

Less formally, given a code C with parity-check matrix Ji and corresponding Tanner graph 
an M-cover of ^ is a graph whose vertex set consists of M copies of Ui and M copies of 
Vj, such that for each j E J, i E X,, the M copies of Ui and the M copies of Vj are connected 
in an arbitrary one-to-one fashion, with edges labelled by the value Hj i. 

For any M > 1, a graph-cover pseudocodeword is a labelling of vertices of the M-cover 
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graph with values from UK such that all parity-checks are satisfied. We denote the label of Uij by 
Pi^i for each i el, i = 1, 2, ■ ■ ■ , M, and we may then write the graph-cover pseudocodeword 
in vector form as 



P = (Pl,l,Pl,2, ■ ■ ■ ,Pl,M,P2,l,P2,2, ■ ■ ■ ,P2,M, " " " ,Pn,l,Pn,2, " " " ,Pn,M) ■ 

It is easily seen that p belongs to a linear code C of length M n over defined by an Mm x Mn 
parity-check matrix Ti. To construct Ti, for 1 < < M and i E X, j e J , we let i' = 

{i - 1)M + i\f = (j - l)M + j*, and so 



Hj^i if Uu* e Mivjj*) 
otherwise 



It may be seen that Q is the Tanner graph of the code C corresponding to the parity-check matrix 

n. 

We also define the n x q graph-cover pseudocodeword matrix 



where 

= \{ie{l,2,--- ,M} : a, = a}| > , 

for i G X, a G 91, i.e. m-"^ is equal to the number of copies of Ui which are labelled with 
a, for each i G X, « G 9^. The normalized graph-cover pseudocodeword matrix is defined as 
(1/M) ■ V. This matrix representation is similar to that defined in [[T6| . Note that the i-th row of 
the normalized graph-cover pseudocodeword matrix (for i E T) can be viewed as a probability 
distribution for the i-th coded symbol q G 9^, in a similar manner to the case of the normalized 
LP pseudocodeword matrix. 

Another representation, which we shall use in Section UXl is the graph-cover pseudocodeword 
vector m = (m,j)jgi where rrii = {'mf'^)aem- for each i G X. Correspondingly, the normalized 
graph-cover pseudocodeword vector is given by (1/M) ■ m G R'^^^^-'". 

It is easily seen that for any c G C, the labelling of Uj ^ by the value Cj for all i E I, 
i = 1,2, ■■ ■ ,M trivially yields a pseudocodeword for all M-covers of ^, M > 1. However, 
non-trivial pseudocodewords exist in general. 
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Example 7.1: To illustrate these concepts, a graph-cover pseudocodeword in shown in 
Figure [2] for the example [4, 2] code over Z3 defined by the parity-check matrix HI). Here the 
degree of the cover graph is M = 4, and we have 



P 



;il22|1122|0011|0011) 



and the parity-check matrix of the code C is given by 



n 



\ 



0010 


2 


2 


10 


1 


2 


2 


10 


10 


2 


2 


10 


10 


2 


2 


1 


2 





10 


2 


2 





10 


2 


2 





10 


2 


2 





1 


2 



V 



Also, the graph-cover pseudocodeword matrix corresponding to p is 

/ 2 2 \ 

2 2 
2 2 
\ 2 2 / 

and the normalized graph-cover pseudocodeword matrix is 

I- 

The graph-cover pseudocodeword vector corresponding to p is 

m=(22|22|20|20) , 

and the normalized graph-cover pseudocodeword vector is 

1 

- ■ m . 



(38) 
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Vl,l (2 110) 




«2,4 (110) 



Fig. 2. Cover graph of degree 4 and corresponding graph-cover pseudocodeword for the example [4, 2] code over Z3 with 
parity-check matrix given by Q. Edge labels are shown in square brackets, and vertex labels in round brackets. This graph-cover 
pseudocodeword corresponds to the LP pseudocodeword described by (I32t-(I35I) via the correspondence described in the proof 
of Theorem 17.11 



B. Equivalence between LP Pseudocodewords and Graph-Cover Pseudocodewords 

In this section, we show the equivalence between the set of LP pseudocodewords and the set 
of graph-cover pseudocodewords. The result is summarized in the following theorem. 

Theorem 7.1: Let C be a linear code over the ring 91 with parity-check matrix Ti and corre- 
sponding Tanner graph Q. Then, there exists an LP pseudocodeword [h, z) with pseudocodeword 
matrix H if and only if there exists a graph-cover pseudocodeword for some M -cover of Q with 
the same pseudocodeword matrix. 

Proof: 

1) Let {h,z) be an LP pseudocodeword of C, and let Q = {V,£) be the Tanner graph 
associated with the parity-check matrix H. We construct an M-cover Q = (y,£), where 
M = J2bec ^j'b, and corresponding graph-cover pseudocodeword, as follows. We begin 
with the vertex set, which consists of M copies of Ui, i E X, and M copies of vj, j E J . 
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Then we proceed as follows: 

• Label h^^'' copies of Ui with the value a, for each z G X, a G 9^. By (|3T1) . all copies 
of Ui are labelled. 

• Label 2^ 5 copies of Vj with the value 6, for every j E J', b E Cj. By (ITTI) . all copies 
of f j are labelled. 

• Next, let t/"-* denote the set of copies of Ui labelled with the value a, for i E I, 
a E Also, for all i E I, j E J, a E 91, let i?-"^ denote the set of copies of Vj 
whose label satisfies 6, = a. The vertices in t/"'' and the vertices in i?-"-* are then 
connected by edges in an arbitrary one-to-one fashion, for every j E J', i E 2j, a E 
All of these edges are labelled with the value Hj^i. 

First, we note that this is possible because 

bGCj, bi=a 

- I 

for every j E J', i E Ij, a E ^K. Here we have used (l26l)). 

Second, we note that all checks are satisfied by this labelling. For j E J , consider 
any copy of Vj with label h. By construction of the graph, the sum, over all vertices 
in M{vj), of the vertex labels multiplied by the corresponding edge labels is 

which is zero because h E Cj. Therefore, this vertex labelling yields a graph-cover 
pseudocodeword of the code C with parity-check matrix H. 
2) Now suppose that there exists a graph-cover pseudocodeword corresponding to some M- 
cover of the Tanner graph Q of C. Then, 
. Step 1: for every i E I, and for every a E we define /i-"^ to be the number of 
copies of Ui labelled with the value a. 

• Step 2: for every copy of vj, j E J , label the copy with the word 6, where hi is equal 
to the label on the neighbouring copy of Wj, i G X,. Then, for every j E J', b E Cj, 
we define Zj^b to be the number of copies of Vj labelled with the word b. 
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Step 2 ensures that zj^b are nonnegative integers for all j E J' and b E Cj, and that (1271 ) 
holds. Also, to show that (|26|) holds, we reason as follows. The right-hand side of (|26|) 
counts the number of copies of vj whose labels b satisfy hi = a. By step 2, this is equal 
to the number of copies of Ui labelled with a, which by step 1 is equal to the left-hand 
side of (l26l) . Therefore, [h, z) is an LP pseudocodeword of the code C with parity-check 
matrix H. 

■ 

As an illustration of the correspondences described in this proof, consider the example [4, 2] 
code over Z3 defined by the parity-check matrix ([4]). First, note that the LP pseudocodeword 
of (|32l)-(|35]) and the graph-cover pseudocodeword of Figure [2] have the same pseudocodeword 
matrix, via (l36l) and (l38l) . Indeed, the reader may check that each pseudocodeword may be 
derived from the other using the correspondences described in the proof of Theorem 17. 1[ 

The next corollary follows immediately from Theorem 17.11 

Corollary 7.2: Let C be a linear code over the ring 9^ with parity-check matrix Ji and 
corresponding Tanner graph Q. Then, there exists a (normalized) LP pseudocodeword (/i, z) if 
and only if there exists a graph-cover pseudocodeword for some M-cover of Q with (normalized) 
graph-cover pseudocodeword vector h. 

Note that this corollary contains two different equivalences, one for normalized objects and the 
other for non-normalized ones. 

VIIL Alternative Polytope Representation 

In this section, we present an alternative polytope for use with linear-programming decoding. 
This polytope may be regarded as a generalization of the "high-density polytope" defined in [|7]|. 
As we show in this section, the new polytope may under some circumstances yield a complexity 
advantage over the polytope of Section Unl In the sequel, we will analyze the properties of this 
polytope. 

First, we introduce some convenient notation and definitions. Recall that the ring 9^ contains 
g — 1 non-zero elements; correspondingly, for vectors h E W^^, we adopt the notation 
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Now, for any j e J , we define the mapping 

b ^ 

defined by 

{Kj{b))a = \{i e Ij : bi-nj^i = a}\ 
for all a E 9^^. We may then characterize the image of nj, which we denote by Tj, as 

rj = Ike : ^ a ■ A;a = and ^ ka< dji , 

for each j E J', where, for any k E N, a E ^i, 

if A; = 

a + ■ ■ ■ + a if k > (k terms in sum) 
Note that Kj is not a bijection, in general. We say that a local codeword b E Cj is fc-constrained 
over Cj if Kj{b) = k. 

Next, for any index set F C X, we introduce the following definitions. Let = |r|. We define 
the single-parity-check-code, over vectors indexed by F, by 



a ■ k 



Cr = |a = (a,),er G : > ^ = }> . (39) 
Also define a mapping Kr : Cr — > by 

(Kr(a))« = |{2 G r : = a}| , 

and define, for k E Tj, 

= {a E C-p : nr{0') = k} . 

Below, we define a new polytope for decoding. Recall that y = (yi, ?/2, ■ ■ ■ , Vn) G S" stands 
for the received (corrupted) word. In the sequel, we make use of the following variables: 

• For alii El and all a E 91^, we have a variable fj°'\ This variable is an indicator of the 
event yi = a. 

• For all j E J and k E Tj, we have a variable fc- Similarly to its counterpart in [7J, this 
variable indicates the contribution to parity-check j of fc-constrained local codewords over 
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• For all j e J, i G Ij, k e Tj, a e yi , we have a variable zlj^.. This variable indicates 

the portion of f^"'^ assigned to fc-constrained local codewords over Cj. 
Motivated by these variable definitions, for all j E J' we impose the following set of 
constraints: 

VzGJ,,VaGlH-, /•^"^ = E4i- (40) 
fcer, 

Vfce7;-,VaGlH-, Yl 4Zk = ka-a,,k. (42) 

i€lj, I3€m.- , I3nj,i=a 

V« e J, , Vfe e 7^- , Va G , > . (43) 

VzGJ^VfcGT;, J] 5^ 45.<^.>- (44) 

We note that the further constraints 

G X, Va G 9^-, < //°^ < 1 , (45) 
Vj ej,\/ke Tj, < (jj- fc < 1 , (46) 

and 

yj ej,\/te I J ,ykeTj,yaE^-, z\"^l^ < a,, fc , (47) 

follow from constraints (l40l)- (|44l) . We denote by U the polytope formed by constraints (l40l)- (|44l) . 

Let T = maxjgj \Tj\. Then, upper bounds on the number of variables and constraints in this 
LP are given by n{q - l)+m{d{q - 1) + l)r and m{d{q - 1) + l)+m{{d+ l)(g - 1) + d)T, 
respectively. Since T < ('^^^~^), the number of variables and constraints are 0{mq ■ d'^), which, 
for many families of codes, is significantly lower than the corresponding complexity for polytope 
Q. 

For notational simplicity in proofs in this section, it is convenient to define a new set of 
variables as follows: 

VjG J,V^GJ„VfcG7;,VaG^-, rg^ = ^ zgj^ . (48) 
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Then constraints (l42l) and (l44l) may be rewritten as 

Vj G J, fc G T;-, Va G Yl = ^" ■ ^J-fc • (49) 

and 

Vj G J, Vz G J„ Vfc G T;-, < 5^ < a,, . (50) 

Note that the variables r do not form part of the LP description, and therefore do not contribute 
to its complexity. However these variables will provide a convenient notational shorthand for 
proving results in this section. 

We will prove that optimizing the cost function ^ over this new polytope is equivalent to 
optimizing over Q. First, we state the following proposition, which will be necessary to prove 
this result. 

Proposition 8.1: Let M G N and k G W^'^. Also let TCI. Assume that for each « G 9^^, 
we have a set of nonnegative integers X^"^ = {xf^^ : i G F} and that together these satisfy the 
constraints 

J24''^ = kaM (51) 

for all a G IH", and 

J2 2^!°^ < M (52) 

for all i G F. 

Then, there exist nonnegative integers : a G Cp*"^! such that 
1) 

J2 Wa = M. (53) 

2) For all a G fH", i G F, 

(k) 

aSCp , ai=a 

The proof of this proposition appears in the Appendix. We now prove the main result. 
Theorem 8.2: The set U = {f : 3 cr, z s.t. {f,a,z) G U} is equal to the set Q = {f : 
3 w s.t. (/, w) G Q}. Therefore, optimizing the linear cost function ([5]) over U is equivalent to 
optimizing over Q. 
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Proof: 

1) Suppose, (/, w) e Q. For all j E J',k e Tj, we define 

beCj, K.j(b)=k 

and for all j E J', i E Ij, k E Tj, a E 9^^, we define 

(a) \^ 

baCj, Kj{b)=k, bi=a 

It is straightforward to check that constraints (l43l) and (|44)) are satisfied by these definitions. 

For every j E J , i Elj, a E we have by (flOl) 

bSCj , bi=a 

fcGT^ beCj, Kj{b)=k, bi=a 

fcer, 

and thus constraint (|40|) is satisfied. 
Next, for every j G J", we have by ^ 

bee, 

= Y Y ^i.^ 

keTj beCj,nj(b)=k 

= Y ' 

fcer, 

and thus constraint (HT)) is satisfied. 
Finally, for every j E J, k E Tj, a E 

Y 

= Y Y 
= Y Y 

beCj , K,j{b)=k ieXj , biHj,i=a 

Y ^i'b 
beCj, Kj{b)=k 

/uq, ■ Cfj^k ■ 
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Thus, constraint (142)) is also satisfied. This completes the proof of the first part of the 
theorem. 

2) Now assume (/, cr, z) is a vertex of the polytope U, and so all variables are rational, as 
are the variables r. Next, fix some j E J',k E Tj, and consider the sets 



for a E ^ .By constraint (|50l) . for each a E , all the values in the set are rational 
numbers between and 1. Let /i be the lowest common denominator of all the numbers 
in all the sets A^^^ a G ^H". Let 



for each a E The sets X^"'^ consist of integers between and /i. By constraint (|49l) . 
we must have that for every a E 9^", the sum of the elements in is equal to /cq/x. 
By constraint (|50l) . we have 



for all i E Ij. 

We now apply the result of Proposition 18.11 with T = X,, M = jj, and with the sets 
X^"-^ defined as above (here = dj). Set the variables {wa '■ a E C^!"^} according to 
Proposition 18.11 

Next, for k E 7^, we show how to define the variables {w'fj : b E Cj, Kj(b) = k}. 
Initially, we set w'^^ = for all b E Cj, i^j{b) = k. Observe that the values (i ■ zf^Joj^^ 
are nonnegative integers for every i E 2, j E J', k E Tj, (3 E 9^^. 

For every a E we define Wa words b^''\ 6*^^\ ■ • • , b^^°''^ e Cj. Assume some ordering 
on the elements f3 E satisfying /3Hj^i = ai, namely /5i,/52, ■ ■ ■ , for some positive 
integer £o. For i E Ij, bf' = 1, 2, ■ ■ ■ , Wa) is defined as follows: bf^ is equal to /3i for 
the first /i-4^'fc/fTj,fc words b^^\b^'^\ ■ ■ ■ , b'^'""'^; bf^ is equal to ^2 for the next j^- z^^ll/aj^k 
words, and so on. For every b E Cj we define 
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Finally, for every b E Cj,Kj{b) = k, we define 



Using Proposition 18.11 



i,j,k V ^ 



/i- 



and so all b^'^\b^'^\ ■ ■ ■ , b^""^) (for all a e 4''^) are well-defined. It is also straightforward 
to see that 6*-^^ G Cj for £ = 1,2, ■■ ■ ,Wa- Next, we check that the newly-defined Wj^b 
satisfy ©-GO]) for every j E J, be Cj. 

It is easy to see that Wj^b > 0; therefore ([8]) holds. By Proposition 18. II we obtain 

beCj, Hj{b)=k 

for all j e J,k eTj, and 

(a) \^ 

b£Cj, K,j{b)=k, bi'Hj,i=a 

for all j G J', i elj, k eTj, a e^l'. Let jSHj^i = a. Since 



/3 : /3Hj,,=a 



by the definition of Wj^b it follows that 



beCj , K.(b)=k, bi=f3 i,j,k beC^ , K.{b)=k, bi'HjA=a 

_ J/3) 

where the first equality is due to the definition of the words b^^\ i = 1, 2, ■ ■ ■ , Wa- 
By constraint (|4T1) we have, for all j G JT", 

keTj beCj, K.j{b)=k 
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thus satisfying 



Finally, by constraint (|40|) we obtain, for aW j E J',i E Ij, /3 E , 



keTj beCj, K.j{b)=k, bi=f3 



thus satisfying (fTOl) . 

■ 

IX. Cascaded Polytope Representation 

In this section we show that the "cascaded polytope" representation described in [|2T]| . ||22]| 
and ll23l can be extended to nonbinary codes in a straightforward manner. Below, we elaborate 
on the details. 

For j E J, consider the j-th row Tij of the parity-check matrix Ti over 9^, and recall that 



Assume that Xj = {ii,i2,--- ,idj} and denote Cj = {1,2,- ■■ ,dj — 3}. We introduce new 
variables 



We define a new linear code of length 2dj — 3 by the {dj — 2) x {2dj — 3) parity-check 
matrix JF^ associated with the following set of parity-check equations over 9^: 




and denote 



X = iX^)jej ■ 



1) 



J _ 



. 



(55) 



2) For every £ = 1, 2, ■ ■ ■ 




. 



(56) 
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Fig. 3. Example of the Tanner graph of a local code Cj = {(feij bi^ bi^ bi^ bi^ ) : bi-^ + 2bi^ + 2bi^ + bi^ + hi^ + 2hi^ = 0} 
of length dj — 6 over $H — Z3, and its transformation into the Tanner graph of the corresponding code Cj^'. Note that the 
degree of each parity-check vertex in the transformed graph is equal to 3. 



3) 

- Xd,-3 + + ^i^.'^JM, = ■ (57) 

We also define a linear code C^^^ of length n + Ylij&jidj — 3) defined by the i^j^j{dj — 
2)) X (n + J2j(^jidj — 3)) parity-check matrix JF associated with all the sets of parity-check 
equations (|55l) -(l57l) (for all j G JT"). We adopt the notation b = {b \ X'') for codewords of Cj^\ 
and c= {c\ x) for codewords of C^^^ 

Example 9.1: Figure [3] presents an example of the Tanner graph of a local code 

Cj = hi^ bi^ bi.^ big) : + 26^2 + 2bi.^ + bi^ + + 26ig = 0} 

of length dj = 6 over 9^ = Z3, and the Tanner graph of the corresponding code Cj^^ of length 9 
(three extra variables were added). The degree of every parity-check vertex in the Tanner graph 
of C]^-* is at most 3. 

The following theorem relates the codes Cj and Cj^\ 

Theorem 9.1: The vector b = {bi)i^Xj G is a codeword of Cj if and only if there exists 
a vector x^ ^ DI'^j-^ such that (6 | x^) ^ C^^ . 
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Proof: 



1) 



Assume b = (foi)ieij G Cj. Define 





bi^Hj^i^ if £ = 1 



if 2 < e < di - 3 



(58) 



Then, obviously, (l55l) holds, and (l56l) holds for all 1 < £ < dj—A. Finally, (l57l) follows from 
subtraction of (l55l) and (l56l) (for each 1 < £ < rfj —4) from the equation XliGX bi-Hj^i = 0. 
Therefore, (b | x') ^ Cj^^ required. 
2) Now, assume that b = (bi)ieXj is such that (6 | x-') G Cj'^-' for some x-' G D^'^j"^, and 
thus (l55])-(l57]) hold (in particular, ^ holds for all 1 < £ < dj - 4). We sum all the 
equalities in (|55l) - (|57l) and obtain that Y^ieij bi ■ T^j,i = 0. Therefore, b e Cj. 

■ 

Note that from this theorem we may see that for every b E Cj, there exists a unique = 
such that b = {b \ x') ^ via (|58l) : we may therefore use the notation 6(6) = (6 | X-'(6)) 
to denote this unique completion, where x^(6) = {Xi{b))ieCj- 

It follows from Theorem 19.11 that the set of parity-check equations (|55l) -(l57l) for all j E J 
equivalently describes the code C. This description has at most + m ■ (rf — 3) variables and 
m ■ {d — 2) parity-check equations. However, the number of variables participating in every 
parity-check equation is at most 3. Therefore, the total number of variables and of constraints 
in the corresponding LP problem (defined by constraints (|8l)-(fT0l) applied to the parity-check 
matrix JF) is bounded from above by 



respectively. 

In the sequel, we make use of some new notations, which we define next. First of all, with 
each parity-check equation prescribed by the matrix JF, we associate a pair of indices 
j G J', i = 1, 2, ■ ■ ■ ,dj — 2, where j indicates the corresponding parity-check equation in H, 
and i indicates the serial number of the parity-check equation in the set of equations (I55l)-(l57l) 
corresponding to the j-th row of H. Denote by Ij^i C X, and Cj/ C Cj the sets of indices i of 



(n + m{d - 3))(g - 1) + m{d - 2) ■ 



and 



m(rf-2)(g' + 3g-2) , 
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variables bi and xl ■• respectively, corresponding to the non-zero entries in row (j, £) of JF. Then, 
each row of JF defines a single parity-check code Cj^^ . For any g G Cj^^^ , we adopt the notation 
9 = {9^ \9^) where 

9^^ = i9i)iei,,i ; 9"" = i9f)iec,,i ■ 

We denote by S the polytope corresponding to the LP relaxation (f8l)-(fT0l) for the code C^^^ with 
the parity-check matrix JF. Recall that codewords of C^^^ are denoted c = (c | It is natural 
to represent points in S as {{f,h),z), where / = U'i"^)iex, aem- and h = {h'j^^)j<.j^i<.c,,aem- 
are vectors of indicators corresponding to the entries Cj {i G X) in c and xl U ^ J ^ ^ l^j) in 
X, respectively. Here 

is a vector of weights associated with each parity-check equation (j, t) and each codeword 
9 e C^. 

Similarly, for each j E we denote by Sj the polytope corresponding to the LP relaxation ([8])- 
(flOl) for the code Cj^\ defined by the parity-check matrix JF^. Recall that codewords of Cj^^ are 
denoted b = (6 | X'')- Then, it is also natural to represent points in Sj as ((/j, hj), Zj), where 
fj = )ieXj, aeiK- and hj = {h^°'i)i(zc^, aem- are vectors of indicators corresponding to the 
entries h {i G Ij) in h and Xi G -^i) in X-', respectively. Moreover, 

is a vector of weights associated with each parity-check equation (j, tj and each codeword 
9 e C^. 

For each j G J", define the mapping analogously to the mapping H with respect to the 
dimensionality of the code Cj^\ namely 

Ej : fR^dj-a ^ ]^|(g-l)(2d,-3) ^ ^iq-l)i2d,-3) ^ 

such that for 6 = (b I x-') G C^^\ 

= I m.) I ■ ■ ■ I m.^) I I ^(xD I • ■ ■ I c(xi.-3)) • 

The next lemma is similar to one of the claims of Proposition 10 in (Si. 
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Lemma 9.2: Let C be a code of length n over 9^ with parity-check matrix H, and let QiH) 
be the corresponding polytope of the LP relaxation, i.e. the set of points (/, w) satisfying dS])- 
(flOl ). Let QiH) denote the projection of Q onto the / variables, i.e. 

Qin) = {f:3ws.t. if,w)EQ} 

Denote by ^ the set of normalized graph-cover pseudocodeword vectors associated with Ti. 
Then, Q{li.) = ^, where *p is the closure of *p under the usual (Euclidean) metric in M^"?"^)". 

Proof: Generally, the proof is similar to the proof of the relevant parts of Proposition 10 
in BH. It is largely based on the equivalence between the set of graph-cover pseudocodewords 
and the set of LP pseudocodewords (Theorem 17.11 and Corollary 17.21) . We avoid many technical 
details, and mention only the main ideas. The proof consists of proving two main claims. 

1) ^ c Q{n). 

Given any normalized graph-cover pseudocodeword vector / G ^, by Corollary 17.21 there 
must exist w with {f,w) E QCH). Therefore / G QCH). 

2) If a point in Q(H) has all rational entries, then it must also be in ^. 

The proof follows the lines of the proof of Lemma 56 in Let {f,w) E QiH) be a 
point such that all entries in / are rational. Then for all j E J , the vector j ^ = (/Jjgi^ 
lies in the convex hull K,{Cj). For convenience in what follows, denote the index set 
^ = {1, 2, ■ • • , (g - l)n + 1}. Using Caratheodory's Theorem [|26l p. 10], for all j E J 
we may write / = jji^^^fP^^^ where /x^-'^ = {iJL^p)iiz^, is a row vector of length |^| whose 
elements sum to unity, and P'--'^ isa|\E'|x(|\E'| — 1) matrix such that for each i E^ , the 
i-th row of P^^\ denoted , satisfies = S(c) for some c E 9^" with Xj{c) E Cj. 
Therefore, 

(/ i) = m(^-)(p(^-) 1), 

where 1 denotes a vector of length |\1'| all of whose entries are equal to 1, is a |\E'| x 
|\1'| system; therefore by Cramer's rule the solution for ^^^^ has all rational entries (this 
argument applies for every j E J). Let M denote a common denominator of all variables 
in vectors for j E J . Define /i^"'' = Mf^°'^ E M for each i E I, a E 9^^ (it is easy to 
see that these variables must be nonnegative integers). Also define = M/xp^ for each 
iE"^, j E J, and 6^^^ = (5?'')ie*- We then have 

h = d^^^p'^j) . (59) 
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Next define, for all j e J, b e Cj 




: pY'=S{c), Xj(c)=b 



By comparing appropriate entries in the vector equation (l59l) . we obtain 




and so (/i, z) is an LP pseudocodeword (the preceding equation yields (l26l) . and (|27]) 
follows from the fact that the sum of the entries in d'^^^ is equal to M for all j G J, these 
entries being nonnegative integers). So the construction of Theorem 17. 1[ part (1), yields a 
corresponding graph-cover pseudocodeword with graph-cover pseudocodeword vector h. 
Therefore the corresponding normalized graph-cover pseudocodeword vector is /, and so 
we must have / G 

The claim of the lemma follows. ■ 
The following proposition is a counterpart of Lemma 28 in [[Si. 

Proposition 9.3: Let C be a code of length n over !H with parity-check matrix H. Assume 
that the Tanner graph represented by 7i is a tree. Then, the projected polytope Q(H) of the 
corresponding LP relaxation problem is equal to /C(C). 

Proof: The proof follows the lines of the proof of Lemma 28 in [HI. Let Q be the labeled 
Tanner graph of the code C corresponding to H. Let Q be an M-cover of Q for some positive 
integer M. Since ^ is a tree, ^ is a collection of M labeled trees which are copies of Q. Let C 
be a code defined by the parity-check matrix corresponding to this Q. We obtain that 



Then, it is easy to see that the set of normalized graph-cover pseudocodeword vectors of H, ^, 
is equal to /C(C) n Q(«-i)". 

To this end, we apply Lemma 19.21 to see that 



C 




X. 



n.m 



) G C for all m = 1, 2, ■ ■ ■ , M 



Q{n) = <p = /c(C) n Q('?-i)" = /c(C) , 



as required. 
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By taking C = Cf' and = Tj so that Q(7Y) = (for j G J"), we immediately obtain the 
following corollary: 

Corollary 9.4: For j G J", let 

= {(/,, ^) : 3 % s.t. ((/^, h,), z,) G 5,} 

Then S.J = /C(C]'^^). 

The proof of the next theorem requires the following definition. Let b G Cj, and let g G Cj^^. 
We say that g coincides with b, writing ^ ixi b, if and only if = hi for all i G Xj^^ and 
gf = xiib) for all i G Cj^e. 
Theorem 9.5: The set 

S = {f : 3 h, z s.t. ((/, h),z) G 5} is equal to the set Q = {f : 3 w s.t. (/, w) G Q}, and 
therefore, optimizing the linear cost function Q over S is equivalent to optimizing over Q. 
Proof: 

1) Let f E Q. Then, there exists w such that (/, w) G Q. Therefore, 

In addition, the entries in w satisfy ^ and 

We set the values of the variables Zj/^g as follows: 

VjGX V^= 1,2,- 2, V^gC]J\ z,-,,g= 5^ wj,f 
So we have that 
Vj e J, yi= 1,2,- ■■ ,dj- 2, Vi G J,- £, Va G 9^-, 

using (|60l) . since Xj^ C for all £ = 1, 2, ■ ■ ■ , — 2. In addition, we define the variables 
h^"} as follows. 

Vj G JT, Vi G Va G 9^-, /ij;^^ = Yl "^i'b . (62) 
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Note that all variables hy'/ are. well defined. It then follows that 
Vj e J, \/i= 1,2,- ■■ ,dj- 2, Vi G Cj^i, Va e 9^-, 

using (l60l) . since C Cj for all £ = 1, 2, ■ ■ ■ , (ij — 2. 
Next, we claim that 

((/,/i),z) g5. (64) 

In order to show this, it is necessary to show (f8l)-(fT0l) with respect to ((/, h), z) and the 
code C^^\ However ^ and ^ follow easily from the definition of the variables Zj^^^g and 
the properties of the variables 5. As to (flOl) . it follows from the combination of (|6TI) 
and (|63]) . 

Finally, (l64l) yields that / G 5, as required. 
2) Now, assume that f E S. This means that there exist h,, z such that ((/, h), z) G S. Then, 
for all j G JT, ((/j-, /ij), %) G By Corollary IM {fj, ^i) lies in X:(C]^^). Therefore, 

Cfph,)= ^ %-S,(6), (65) 

where ^^^^(x) 5 = 1 and P^-^ > for all b G C]^\ 
For all j E J', b E Cj set the value of wj b as 

Wj^b = f^j,b{b) ' 

and thus 

Y '^j'b = 1 , (66) 

and 

b > for all bECj . (67) 

Then, (l65l) becomes 

Comparing the first set of coordinates, we obtain that 

baCj, bi=a 
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This set of equations holds for all j E J . Together with (|66l ) and (|67] ) this means that 
{f,w) e Q. Therefore, / G Q, as required. 

■ 

The polytope representation described in this section leads to a polynomial-time decoder for 
a wide variety of classical nonbinary codes (for example, generalized Reed-Solomon codes). 



A. Comparison with ML Decoding 

In this section we compare performance of the linear-programming decoder with hard-decision 
and soft-decision based ML decoding. For such a comparison, a code and modulation scheme 
are needed which possess sufficient symmetry properties to enable derivation of analytical ML 
performance results. We consider encoding of 6-symbol blocks according to the [11,6] ternary 
Golay code, and modulation of the resulting ternary symbols with 3-PSK modulation prior to 
transmission over the AWGN channel. Figure |4] shows the symbol error rate (SER) and codeword 
error rate (WER) performance of this code under LP decoding using the polytope Q of Section 
Uni Note that this is the same as its performance using the polytope U of Section IVIIIl and its 
performance using the polytope S of Section |Kl When the decoder reports a decoding failure, 
the SER and WER are both taken to be 1. To quantify performance, we define the signal-to-noise 
ratio (SNR) per information symbol 7^ = Es/Nq as the ratio of the received signal energy per 
information symbol to the noise power spectral density. Also shown in the figure are two other 
performance curves for WER. The first is the exact result for ML hard-decision decoding of the 
ternary Golay code; since the Golay code is perfect, this is obtained from 



where p{^s) represents the probability of incorrect hard decision at the demodulator and was 
evaluated for each value of 7^ using numerical integration. The second WER curve represents 
the union bound for ML soft-decision decoding. Using the symmetry of the 3-PSK constellation, 
this may be obtained from 



X. Simulation Study 
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Fig. 4. Codeword error rate (WER) and symbol error rate (SER) for the [11,6] ternary Golay code with 3-PSK modulation over 
the AWGN channel. The figure shows performance under LP decoding, as well as the exact result for hard-decision decoding 
and the union bound for soft-decision decoding. 
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where R{C) = 6/11 denotes the code rate, and the Hamming weight of the codeword c E C, 
wh{c), is distributed according to the weight enumerating polynomial [|27l 

W{x) = l + 132x^ + 132x^ + 330x^ + llOx^ + 24a; 
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The performance of LP decoding is approximately the same as that of codeword-error-rate 
optimum hard-decision decoding. The performance lies 0.1 dB from the result for ML hard- 
decision decoding and 1.53 dB from the union bound for codeword-error-rate optimum soft- 
decision decoding at a WER of 10^''. These results are comparable to those of a similar study 
conducted for the binary case in [TJ. 

B. Low-Density Code Performance 

Figure [5] shows SER and WER simulation performance results for two low-density parity- 
check (LDPC) codes. The first code C'^^\ of length n = 150, is over the ring 9^ = Z3, where 
nonbinary coded symbols are mapped directly to ternary PSK signals and transmitted over an 
AWGN channel, the mapping described in Example 15.31 being used for modulation. The parity- 
check matrix H^^^ consists of m = 60 rows and is equal to the right-circulant matrix 

1 if i -j G {0,51,80} 

2 if i -J G {8,30,90} 

otherwise. 

The code rate is R{C^^^) = 0.6. As expected, the performance of the low-density code C^^^ is 
significantly better than that of the ternary Golay code given in Figured The second code C*^^\ 
of length n = 80, is over the ring UK = Z4, where nonbinary coded symbols are mapped directly 
to quaternary phase shift keying (QPSK) signals and transmitted over an AWGN channel, the 
mapping described in Example 15.31 again being used for modulation. The parity-check matrix 
Ti*^^^ consists of m = 32 rows and is equal to the right-circulant matrix 

1 if i -j G {0,41,48} 

3 ifi-je {8, 25} 
otherwise. 



-1/(1) 



This code also has rate R(C^'^^) = 0.6. The quaternary code has a higher SER and WER than the 
ternary code for the same Es/Nq; however it has a smaller block length and a higher spectral 
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Fig. 5. Codeword error rate (WER) and symbol error rate (SER) for the [150, 90] ternary LDPC code C'^^ under ternary PSK 
modulation, and for the [80, 48] quaternary LDPC code C'^' under QPSK modulation. 



efficiency. In both systems, when the decoder reports a decoding failure the SER and WER are 
both taken to be 1. 



XL Future Research 



Sections [Villi and HXl presented two alternative polytope representations, which have a smaller 
number of variables and constraints than the respective standard LP representation in certain 
contexts. It would be interesting to further reduce the complexity of the polytope representation 
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in order to yield more efficient decoding algorithms. Alternatively, one could try to reduce 
complexity of the LP solver for the nonbinary decoding problem by exploiting knowledge of 
the poly tope structure. 

The notion of pseudodistance for nonbinary codes was recently defined in ll28l . and lower 
bounds on the pseudodistance of nonbinary codes under g-ary PSK modulation over the AWGN 
channel were presented. It would be interesting to obtain lower bounds on the pseudodistance 
for other families of nonbinary linear codes and for other modulation schemes. 

Appendix 

Proof of Proposition \8.1\ 

Preliminary to proving this Proposition we give some background material on flow networks. 

Flow Networks: Let G = (V, E) be a directed graph, and let {s, t} C V, s 7^ t. A flow network 
(G(V, E), c) is a graph G = (V, E) with a nonnegative capacity function c : E — > R U {+00} 
defined for every edge. 

For a subset V C V let V" = V\V'. We define a cut (V : V") induced by V as a set of edges 
: M G V, t> G V"}. The capacity of this cut, c(V' : V"), is defined as 

c(V':V")= 

For the edge e = (m, v) we use the notation e G in(t>) and e G out(M). We also use the notation 
7V(f ) to denote the set of neighbors of v, namely 

J\f{v) = {u : {u,v) G E} U {u' : {v,u') G E} . 

For a set of vertices Vq C V, denote 

A/'(Vo) = U,eVoA/'(^^)\Vo . 

The flow in the graph (network) G with a source s and a sink t is defined as a function f : E — > 
M U {+00} that satisfies < f (e) < c(e) for aU e G E, and 

eGE, egin(?j) eGE, eGout(ti) 

The value of the flow f is defined as 

eSE, e6in(t) e€E, e6out(s) 
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The maximum flow in the network is defined as the flow f that attains the maximum possible 
value. There are several known algorithms, for instance the Ford-Fulkerson algorithm, for finding 
the maximum flow in a network, the reader can refer to ||29l Section 26.2]. It is well known 
that the value of the maximum flow in the network is equal to the capacity of the minimum cut 
induced by a vertex set V such that s G V and t ^ V (see [|291l ). 

Finally, we prove the Proposition. 

Proof: The proof will be by induction on M. We set = for all a G C^^\ We show 
that there exists a vector a = (oj) -gp G C^!'^ such that 

(i) For every i eT and a G 9^~, 

tti = a =^ xf^^ > . 

(ii) If for some i G F, Xloe^H- -^i"^ ~ ^^^^ ai = a for some a G 9^^. 

Then, we 'update' the values of x'f^'s and M as follows. For every i G F and a G 9^^ with 
Oj = a we set x["^ x^-"^ — 1. In addition, we set M M — 1. We also set Wa Wa + I- 
It is easy to see that the 'updated' values of x.°^'s and M satisfy 

for all a G and XlaeiR- ^i"^ — ^ alH G F. Therefore, the inductive step can be applied 
with respect to these new values. The induction ends when the value of M is equal to zero. 

It is straightforward to see that when the induction terminates, (|53l) and (|54l) hold with respect 
to the original values of the xf'^ and M. 

Proof of existence of a that satisfies (i): We construct a flow network G = (V, E) as follows: 

v = {s,t}uUiUU2, 

where 

Ui = ^R- and Us = F . 

Also set 

E = {{s,a)}ae<n-' U {(i,t)}ier U {(a, . 
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We define an integer capacity function c : E — > N U {+00} as follows: 

ka if e = (s, a), a G IH^ 

1 if e = {i,t), ieV . (68) 

+00 if e = {a,i), a G i G F 

Next, apply the Ford-Fulkerson algorithm on the network (G(E,V),c) to produce a maximal 
flow imax- Since all the values of c(e) are integer for all e G E, so the values of fmaa;(e) must 
all be integer for every e G E (see ||29l ). 

We will show that the minimum cut in this graph has capacity Cmm = J2a(^m- ^a- First, 
consider the cut induced by the set V = {s}. This cut has capacity J2aeyi- therefore 

Assume that there is another cut, which has smaller capacity. If this smaller cut is induced by 
the set V = V\{t}, its capacity must be > XlaeiH- ~ smaller. Therefore, without 

loss of generality, assume that the minimum cut is induced by the set V, where V = {s} U 
X' U Y', X' C Ui and Y' C U2. Let X" = Ui\X' and Y" = U2\Y' (and so V" = {t} UX" U Y"). 

Observe that there are no edges (a, i) G E with a eX', i E Y", because otherwise the capacity 
of the respective cut would be infinitely large (so it cannot be a minimum cut). Thus, 



Observe also that 



|Y'| > |U2nAr(X')| (69) 



and 



Therefore, 



We obtain that 



|U2nAr(X')| > (70) 



aeX' 



c(V' : V") = 5^ + |Y'| > A;„ + A;, = fc„ , (71) 

oeX" aex" aex' a&<n~ 



where the inequality is due to (1691 ) and (1701) . This leads to a contradiction of the non-minimality 
of c(V' : V") for V = {s}. 
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If we apply the Ford-Fulkerson algorithm (or a similar algorithm) on the network (G(V, E), c), 
we obtain that the integer flow i^ax in G has a value of XlagfR- ^a- Observe that fmaa:((a,^)) £ 
{0, 1} for all a E and i eT. Then, for all i G F, we define 

« if ^maxiio!,i)) = 1 for some a G Ui 
otherwise 

For this selection of a = (ai, 02, ■ ■ ■ , ctv), we have a E C^!"^ and ai = a only if xf'^ > 0. 

Proof of existence of a that satisfies ( i) and ( ii) simultaneously: We start with the following 
definition. 

Definition A.l: The vertex i E U2 is called a critical vertex, if 

In order to have (|52l) satisfied after the next inductive step, we have to decrease the value of 
xl°'^ by (exactly) 1 for every critical vertex. This is equivalent to having fmaxiih t)) = 1. 
We have just shown that the maximum (integer) flow in G has value J^ae^n- Now, we aim 
to show that there exists a flow f* of the same value, which has f*((i,t)) = 1 for every critical 
vertex i. 

Suppose that there is no such flow. Then, consider the maximum flow f, which has f{{i,t)) = 
1 for the maximal possible number of the critical vertices z G U2. In the sequel, we assume that 
there is a critical vertex iq E U2, which has f'((io,i)) = 0. We will show that the flow f can be 
modified towards the flow f" of the same value, such that for f" the number of critical vertices 
z G U2 having f"((i,t)) = 1 is strictly larger than for f. 

Indeed, if there exists vertex «o G A/'(i) such that (acii) G E and f'((Q;o,ii)) = 1 for some 
non-critical vertex ii, then f'((ao7'^o)) = 0, f'((2o,i)) = and f'((2i,t)) = 1. We define the flow 
f" as 

f 

1 if e G {(ao,io), («o,^)} 
f"(e)=<! if eG {(ao,*i),(*i,t)} • 
f'(e) for all other edges e G E 

It is easy to see that f" is a legal flow in (G(V, E), c). Moreover, it has the same value as f, and 
the number of critical vertices z G U2 satisfying f"((i,t)) = 1 is strictly larger than for f. 

50 



In the general case (when there is no vertex ao as above), we iteratively define a maximal set 
Z of vertices a G Ui satisfying the next two rules: 

1) For any a E Ui: if {a, io) E E then a E Z. 

2) For any « G Ui and i E U2: if G E, f'((a,i)) = 0, and there exists P eZ such that 
{(3,i) E E, f((/3,z)) = 1 and all i' E U2 with f'((/3, z')) = 1 are critical, then a E Z. 

Consider the set Z. There are two cases. 

Case 1: Every vertex a in Z satisfies 

Vz G U2 : {a, i) E E and i is not critical =^ f'((tt, i)) = ■ 

Then, for every a G Z there are exactly ka vertices i such that (a, i) E E and f'{{a, i)) = 
1. Define 

T = {z G U2 is critical : 3a G Z s.t. {a,i) E E and f'{{a,i)) = 1} . 
We have 

\J\ = J2i = J2K. (72) 

Note that io and recall that 

iP, io) E E for some p eZ. (73) 

Note also that if 7 ^ Z and i E T, then there is no edge between 7 and i (otherwise, 
f'((7, i)) = 0, and so 7 should be in Z). Therefore, xf^^ = 0, and so 

EE-:°'= EE-.'"'- (74) 

aGZ iGT aGlH- ieT 

We obtain 

E^.M = EE-i°'>EE-l°' 

aez aGZ «er aeZ ieT 

= EE4"' = EE-r' 

ieT aeZ 

Here the first equality is due to (|5T1) . the strict inequality is due to (1731) and the second 
equality is due to (IT?! ). The third equality is obtained by the change of the order of the 
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summation. The fourth equality is true because all vertices in T are critical. Finally, 
the fifth equality is due to (|72|) . 
Therefore, this case yields a contradiction. 
Case 2; There is a vertex «o in Z which satisfies 

3jo e U2, (ao, io) e E, jo is not critical and f'((ao, Jo)) = 1 • 
However, by the definition of Z, there is an integer i and a set of edges 

{{ah,jh+i)}h=o,i,-/ C E and {{at, jh)}h=i,2,- / ^ E , 
such that ji+i = io, 

ah & Z for /i = 0, 1, ■ ■ ■ , £ , 
j;, e U2 for h = l,2,--- , 

and 

f'(K,j^+i)) = Ofor /i = 0,l,--- 
f'(K,j;,) = lfor/i = l,2,---,£. 

We define the flow f " as 

1 if e G {(ah,Jh+i)}h=o,i,-/ U {(j£+i,t)} 
f"fe)=<! if e G {K,jft)U=o,i,...,^ U {(jo,t)} • 
f'(e) for all other edges e 

This f" is a legal flow in (G(V, E),c). Moreover, it has the same value as f, and the 
number of critical vertices i G U2 having f"((i,t)) = 1 is strictly larger than for f. 

We conclude that there exists an integer flow f* in (G(V, E), c) of value XlaeiH- ^o^' ^^''^ 
for every critical vertex i G U2, f*((z,t)) = 1. We define 

a if f*((a,i)) = 1 for some a G Ui 
otherwise 

and a = (ai)jer- For this selection of a, we have a G C^!"^ and the properties (i) and (ii) are 
satisfied. ■ 
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